Data Driven by Jeremy David Curuksu

Author:Jeremy David Curuksu
Language: eng
Format: epub
Publisher: Springer International Publishing, Cham

(7.1)

where A is a square n × n covariance matrix (i.e. the set of all covariances between n variables as obtained from Eq. 6.​1), v is an unknown vector of dimension n, and λ is an unknown scalar. This equation, when satisfied, indicates that the transformation obtained by multiplying an arbitrary object by A is equivalent to a simple translation along a vector v of magnitude equal to λ, and that there exists n pairs of (v, λ). This is useful because in this n-dimension space, the matrix A may contain non-zero values in many of its entries and thereby imply a complex transformation, which Eq. 7.1 just reduced to a set of n simple translations. These n vectors v are the characteristic vectors of the matrix A and thus referred to as its eigenvectors.

Once all n pairs of (v, λ) have been computed,4 the highest eigenvalues indicate the most important eigenvectors (directions with highest variance), hence a quick look at the spectrum of all eigenvalues plotted in decreasing order of magnitude enables the data scientist to easily select a subset of directions (i.e. new variables) that have most impact in the dataset. Often the eigen-spectrum contains abrupt decays; these decays represent clear boundaries between more informative and less informative sets of variables. Leveraging the eigen-decomposition to create new variables and filter out less important variables reduces the number of variables and thus, once again, simplify the prediction problem.

The eigenvector-eigenvalue decomposition is commonly referred to as PCA (Principal Component Analysis [177]) and is available in most analytics software packages. PCA is widely used in signal processing, filtering, and noise reduction.

The major drawback of PCA concerns the interpretability of the results. The reason why I could name the new variables typical and atypical in the example above is that we expect income and education levels to be highly correlated. But in most projects, PCA is used to simplify a complex signal and the resulting eigenvectors (new variables) have no natural interpretation. By eliminating variables the overall complexity is reduced, but each new variable is now a composite variable born out of mixing the originals together. This does not pose any problem when the goal is to reconstruct a compound signal such as an oral speech recorded in a noisy conference room, because the nature of the different frequency waves in the original signal taken in isolation had no meaning to the audience in the first place. Only the original and reconstructed signals taken as ensembles of frequencies have meaning to the audience. But when the original components do have meanings (e.g. income levels, education levels), then the alternative dimensions defined by the PCA might loose interpretability, and at the very least demand new definitions before they may be interpreted.

Nevertheless, PCA analyses remain powerful in data science because they often entail a predictive modeling aspect which is akin to speech recognition in the noisy conference room: what matters is an efficient prediction of the overall response variable (the speech) rather than interpreting how a response variable relates to the original components.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.